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Abstract — Proving capacity for networks without feedback or 
cooperation usually involves two fundamental random coding 
techniques: superposition coding and binning. Although con- 
ceptually very different, these two techniques often achieve the 
same performance, suggesting an underlying similarity. In this 
correspondence we propose a new random coding technique that 
generalizes superposition coding and binning and provides new 
insight on relationship among the two With this new theoretical 
tool, we derive new achievable regions for three classical informa- 
tion theoretical models: multi-access channel, broadcast channel, 
the interference channel, and show that, unfortunately, it does 
not improve over the largest known achievable regions for these 
cases. 

Index Terms — random coding, superposition coding, binning, 
multi access channel, broadcast channel, interference channel. 

I. Introduction 

Apart from few notable exceptions IH, O, the capacity 
region for a general multi-terminal network is shown using 
random coding techniques such as rate splitting, time sharing, 
superposition coding, binning, Markov encoding, quantize, and 
forward and few others. For networks with no feedback or 
cooperation, the two random coding techniques are usually 
considered when proving capacity: superposition coding and 
binning. Superposition coding can be intuitively be thought 
of as stacking codewords on top of each other | 3 | and is 
obtained by generating the codewords of the "top" codebook 
conditionally dependent on the "base" codeword. A typical 
representation of this encoding technique is the one Fig. [T] 
(Si Fig 4] where the codeword 1/2^ is superposed to the 
codewords Uf^ . A codeword in the base codebook is 
randomly selected from the typical set T^(Pf/J. For each base 
codeword, a top codebook is generated by selecting random 
elements from the typical set 7^ (^t/2|t/i)- Superposition 
coding is often thought of as placing spheres, or clouds, in 
the typical set 7^ {PU1U2) ' the base codewords are "cloud 
centers" while top codewords are "satellite codewords". If the 
number of spheres is small enough-a low rate for the base 
codewords-and their size is small enough-a low rate for the 
top codewords-then codewords are sufficiently spaced apart to 
allow successful decoding. 

Binning Q allows a transmitter to "pre-cancel" (portions of) 
the interference experienced at a receiver. A usual representa- 

^ sometimes referred to as Cover's random binning [4J or Gel'fand-Pinsker 
coding |5|. 



Typical Typical set 




Fig. 1. A graphical representation of superposition coding. 

tion of binning (HI Fig. 14.19] is the one in Fig. O here the 
codewords 1/2^ is binned against Uf^ . The codebook of Uf^ 
is generated as in superposition coding while the codewords 
for 1/2^ and are selected from the typical set 7^ {P2) and 
placed in bins. Codewords in the same bin are associated with 
the same message, i.e. multiple codewords can be used to 
communicate the same message. The codeword U2 1^ the bins 
are selected for transmission when it belongs to the typical set 
7^ {Pui,U2)^ ^ven though generated independently from . 
It is possible to find a codeword that satisfies this condition if 
the size of each bin is sufficiently large. Binning is commonly 
interpreted as dividing the typical set 7^ {Pu-^^u^) the 
partitions formed by the bins in which the codewords 
and U2 are placed. Encoding is successful when the size of 
the bins is sufficiently large-large binning rate-while decoding 
is successful if the transmitted codewords are sufficiently far 
apart-low message rate. In certain cases, it is possible to 
simultaneously bin two codewords against each other: this 
coding technique is usually referred to as joint binning. 

Despite the difference in this two random coding techniques, 
in many cases they have identical performance |7, Sec. VI]: 
this suggests an underlying similarity in the way the two 
techniques select the codewords to be transmitted. To gain 
a better understanding of the properties of theses strategies, 
we develop a new random coding technique that encompasses 
superposition coding and binning as special cases. In this 
scheme codewords are first superposed according to a cer- 
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Fig. 2. A graphical representation of joint binning, 
tain distribution, the codebook distribution, and successively 
binned to appear as if generated according to a different 
distribution, the encoding distribution. Classical superposition 
coding corresponds to the case where the binning distribution 
is the same as the codebook distribution while binning is 
obtained when the codebook distribution has independent 
codewords. All the strategy in between these two cases have 
never been previously considered in literature. We use this new 
random coding technique to derive achievable regions for the 
multi-access channel, the broadcast channel, and the interfer- 
ence channel. Unfortunately these new achievable regions do 
not improve on the largest known achievable regions for the 
broadcast channel and the interference channel but show that 
these regions can be obtained with a wider set of encoding 
strategies than what was previously. 

Paper Organization: Section [III introduces the a new ran- 
dom coding techniques that generalizes superposition coding 
and binning. Section Jill presents new achievable regions for 
classical communication models. Section |IV] concludes the 
paper. 

II. Combining Superposition Coding and Binning 

We introduce the new random coding technique that gener- 
alizes superposition coding and binning with a simple exam- 
ple. Consider a classical Broadcast Channel with a common 
message (BC-CM) where two messages Wi and W2 are 
encoded at transmitter 1 , message W2 is decoded at receiver 2 
while message Wi is decoded both decoder 1 and decoder 2. 
The channel outputs {Yi^Y2) are obtained from the channel 
input X from the channel transition probability Py^,Y2\x- 

Take any distribution for the codebook generation and the 
encoding procedure 



(1) 



-'"^codebook — ■^U^U^') -'encoding — u^u^i 

and let Wi ^ [l . . . 2^^^] be the messages Wi to be transmit- 
ted for i e {1,2}. 
• Codebook Generation 

1) Generate 2^^ codewords with N iid draws from the 
distribution P[/c. Index these codewords as Uf^ {wi^bi) for 
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2) For each /7f (^1,61 



Ri + Ri . 

, generate 2^^^ 



codewords 1/2^ 



1...2' 



with 



codewords as /72^ (k;2, ^2, '^1, ^1) for 62 ^ 

L2 = + R2 

• Encoding Procedure 

For each message set {wi^W2), choose the bin indexes 61 and 
62 so that 

{/7f K, 61), (^2,^2,^1,^1)} GTf(Pf/e^e). (2) 

If no such set (61,62) exists, pick two indexes at random. 
Generate the channel input X as a deterministic function of 
the Random Variables (RVs) Ui and U2. 

• Decoding Procedure 

1) Decoder 1 looks for a set of indexes wl^bl such that 

{n^,C/fK,6})}eTf(Pv„t;e) 

2) Decoder 2 looks for a set of indexes wl^wl^bf^ b^ 

{Ff,f/f(5;^6?),f/f(«;?,6?,5;i,6l)}GTf(Py,c;e^|) 

To determine the performance of the achievable scheme 
above we need to determine the encoding and decoding error 
probabilities. 

The key techniques in bounding the encoding error is related 
to the probability that there exists a random vector Ul^ in the 
typical set of certain distribution 7^ {Pu^), that also belongs 
to the typical set of a different distribution 7^{Pu^). The 
probability of this event can be bounded using |8, Lem. 2.6 ]. 

Lemma II.l. Inaccuracy, |[8l Lem. 2.6]. Consider a two 
general distributions Pu2 cind Pu2 (^nd G 7^ {Pu-^), then 

Pff,{u^) = exp {-N{D{Pu,\\Pu,) + H{Ui)} 

The quantity 

D{PuA\Pu,) + H{Ui) = -Y,Pu^{ui)\og{PuAui)) 

Ui 

is referred to as inaccuracy and it can be used to derive a 
more general version of the covering lemma [4J. 

Theorem II.2. Generalized Covering Lemma 

Consider the set_ of RVs Ui and U2 with distribution 
PuiU2 t^ke i.i.d. sequences U2 from the typical set 
7^ {PU2) y indexed as U2 (^), i G {1 . . . 2"^^}, then there exists 
5{e) ^ as e ^ such that 

Ph? G{1...2"^}, s.t. ii^(?) G Tf (Pf/jl ^1, 
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with N iid draws from the distribution Pi 
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Proof: The complete proof is provided in (91. ■ 
With Th. III. 21 we derive the achievable region of the pro- 
posed random coding strategy. 



Corollary II.3. An Achievable Region for the BC-CM The 

following region is a achievable for a general BC-CM 

R^ + R2>D{Plj^^u^\Pk,u.) (3) 
ii < J(ri;/7r) + Z)(Pt/e||Pyc) 

L2 < I{Y2; UI\U!) + D{Pui\\Pui\Put) 
Li + L2< I{Y2; Ul Ul) + D{Pue^ui\\Pu'^u^), 

for Li = Ri -\- Ri for i G {1,2} and any distribution of 

P 3odebook and -Pencoding i^ (ffl)* 

Proof: The complete proof is provided in ■ 

• Superposition coding: is obtained by having the distribu- 
tion imposed at encoding Pencoding equal the distribution of 
the codebook Pcodebook, i.e. 

Puiu^ = Pu^u^' (4) 

In this case the binning rates Ri and R2 can be set to zero, 
thus obtaining the region 

R2<I{Y,;UI\U',) 
Ri^R2<I{Yi;U^,UI), (5) 

• Joint Binning: corresponds to the case where the distri- 
bution of the codebook Pcodebook cquals the product of the 
marginals of encoding distribution: 

Pu^ = Puf Pu^ . (6) 
which results in the region 

^1+^2 >I{U!;UI) 

L2<I{Y2',U^2\Ul) 
Li^L2< I{Y2; [/r, /7|) + I{U!; [/|), (7) 

• Binning: The region with binning is obtained from the 
region with joint binning of above by setting either Ri or R2 
to zero. 

III. Achievable Regions for Classical Channel 

We apply the new random coding technique in SecHIlto the 
Multi-Access Channel with Common Messages (MAC-CM) 
[10], 11 IJ, the Broadcast Channel (BC) 13], (TT\, fU] and the 
InterFerence Channel (IFC) [14J, 1 15|. Capacity is known for 
the MAC-CM and for a subsets of both the BC and the IFC. 
The largest achievable regions in each case can be achieved 
by employing a combination of rate splitting, superposition 
coding and binning (71 Sec. VI]. In the following we adopt the 
notation in fT| to describe the channel model and distribution 
of messages and codewords. In particular, the codeword 
with rate Ri^j, encodes the messages Wi^j from the i set of 
transmitters to the j set of receivers. 



A. The Multi-Access Channel with Common Messages 

In the classical MAC ifTOt . idTl . two transmitters communi- 
cate a message each to a single decoder. In the MAC-CM an 
additional common message is transmitted by each source to 
the decoder 1 16|. Let U^^he the codeword associated with the 
message from transmitter i to receiver 1, fori G {1,2, {1,2}} 
respectively. Using the random coding technique in Sec. [Ill we 
can superpose Ul^^i and /72^2 ^^^^ ^{1 2}^i successively 
bin and against U^^ 2}^i • 

Corollary III.l. An Achievable Region for the MAC-CM 

The following region is achievable for a general MAC-CM: 

Ri^i > D{Pu^^J\Pu^jPu^^^^^^^^) 
R2^i>D{PuU\PuUPuUi.J 

P^ (Pencoding I Pcodebook) 

ii^i + ^2^1 < /m;C/i%i,c/2%i|/'c/|,,,j^,) 

H~ P (Pencoding I Pcodebook) 

ii^i </(n;C/^i|C/|i,2Hi,C/2^i) 

^2^1 </(ri;C/2%i|C/|i,2j^i,C/i%i) 

+ DiPutJ\PuUPu!.^,,,,), 

(8) 

union over all the distributions that factor as 

Pcodebook = ^f/fi,2}^i^f^?^ilt^{i,2}^i^^l^il^fi,2}^i 
Pencoding = ^t/^^,^}^! ^t^!^! I ^^{1,2}^! ^^I^i I i,2}^i ' 

and for Li^j = Ri^j + ^i^j- 

Proof: The complete proof is provided in ||9l. ■ 
After the Fourier-Motzkin Elimination (FME) of the region 
in ©, we obtain the classical region [161 

P{1,2}^1 + ^1^1 + ^2^1 < HYl] ^{1,2}^15 ^1^15 ^2^l) 

i^i^i +i^2^i < I{Yi;U!^,,UI^,\Ul^,y^,) 
Ri^i <HYi'.U!^i\Ul,2}^r.Ul^i) 
i^2^i < /(n;/72%|/7^^,2Hi,/7i%), 

(10) 

union over all the possible distributions in (|9]) which is indeed 
capacity. Cor. IIII.ll shows that the capacity of the MAC- 
CM can be achieved with any distribution of the codewords 
U{^_^i and 112^^1 for as long as the codewords can be further 
binned to impose the distribution of the matching outer bound. 
The distance between the codewords at generation and after 
encoding has no effect on the resulting achievable scheme. 

B. The Broadcast Channel 

In the BC L3J, 111 211 . [|13il one encoder wants to communicate 
to two decoders a message each. For this channel model, rate 
splitting can be applied so as to split each message in private 
and common part; the two common part can then be embedded 
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into a single common message. This transforms the problem 
of achieving the rate vector [^i^ 1,^1^2] the problem of 
achieving the rate vector [i^i^i, i^i^2, ^i^{i,2}] where 



\Rl^l Rl^2 ^1^{1,2}] •> (11) 



for any a G [0 ... 1] and a = 1 — a. The random coding 
technique of Sec. [Ill can be applied to the BC after the rate 
splitting in (fTTl) by superposing U^^^i and /7]^2 ^"^^^ ^iL{i2} 
and successively jointly binning t^i^i, ^^1^2 ^iL{i2}- 

Corollary III.2. An Achievable Scheme for the BC The 

following region is achievable for a general BC: 



with 



^i^{i,2} + Li^i < /(Yi; /7i^{i,2}^r^i) 

^DlPu- t/e ||Pf/c C/c ) 

ii^i </(n;C/^i|f/i^{i,2j) 

^i^{i,2} + ^1^2 < ^(^2; ^r^{i, 2)^1^2) 

V ^1^2^1^{l,2} " ^1^2^1^{1,2} ^ 

il^2</(r2;C/i%2|f^r^{l,2}) 

+ D(P^^^J|P^c^JP^e^^^^^^) (12) 

w^/6>/t over all the distributions that factor as 

Pcodebook = ^t/r^{i,2}^t^r^iif^i^{i,2}^^r^2if^i^{i,2} ^^^^^ 

Pencoding = ^t/i^i t/f^2 ^r^{i,2} ' ^-^"^^^ 

for the rate splitting strategy in (fTTl) and Li^j = R\^-^^R\^y 



Proof: The complete proof is provided in |9|. ■ 
After the FME of the binning rates i^i^j we obtain the 
region 

Ri^i </(ri;C/^i|f/r.{i.2}) 
Ri^{i,2} + Ri^i < /(i^i; f/r^{i.2}f^i-i) 

i?i^{i,2} + Ri^2 < /(I2; c/r^{i,2}f/r^2) 

Ri^{i,2} + Ri^i + Ri^2 < -^(^1; t^r^{l,2}f^l^l) 

+ /(F2;C/r^2|t^r^{l,2})-^BC-RS 

Ri^{i,2} + Ri^i + Ri^2 < /(^2; C/r^{i,2}f^l^2) 

+ C/i^i|C/^{i,2}) - i?BC-RS, (14) 



-DbC-RS = ^ -P(7: 

Ml^l,Ml^2,«l^{l,2} 



.{l,2}'-^l^l'-^1^2 



log 



= -J(C/i^i,/7i%|C7^^i,2})- (15) 

With the equivalence in ([T5]) we conclude that the region in 
([T4l) is equivalent to Marton's region ifTTl which is the largest 
known achievable region for a general BC. As for Cor. IIII.ll 
Cor. nil. 21 shows that the achievable region is not determined 
by the distribution of the codewords in the codebook but only 
on the distribution after encoding. 

C. The Interference Channel 

The IPC is four-terminal network where two pairs of trans- 
mitter/receiver pairs want to communicate a message over the 
channel each. As for the BC, we can rate-split each message 
into a public and private part: the private messages, l^i^i and 
^2^2 respectively, are decoded only at the intended transmit- 
ter while the public messages, I^i^{i,2} and W2^{i,2}. are 
decoded by both decoders. Rate- splitting transforms the prob- 
lem of achieving the rate vector (^1^1, ^2^2) 1^ problem 
of achieving the rate vector (i^i^i, i^2^2, ^i^{i,2}5 ^2^{i,2}) 
where 
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0/3 0^ 



[Rl^l R2^2 Rl^{l,2} ^2^{1,2}] : 

(16) 

for any (a, /3) G [0 . . . 1]2 , a = 1 - a , ^ = 1 - /3. The new 
random coding technique of SecHIlcan be applied to the IPC 
after rate splitting by superposing the /7]^i onto 2} 
jointly binning Ul[^i^U^_^^-^ 2}- same encoding procedure 
is applied to the codewords U2L2 and U^_^^-^ ^j- 

Corollary III.3. An Achievable Scheme for the IFC The 

following region is achievable for a general IFC: 

Sl^{l,2} +^1-1 > I^(Pt/e^,,C/e^^^^J|Pt; 

R,^,>D{Pu^^J\PuUPu^^^^^^,^ 

^2^{l,2} + ^2^2 > D{Pui 

R2^2 > D{Pui^J\Pui 



'^1 ^{1,2} ^ 



Tje WPw^ ) 

■^2^2 ''^2^{1,2} ' ' ^2^2 ''^2^{1,2} ^ 



^2^{1,2}^ 



■1<^1^{1,2} I 



>1^1<^1^{1,2} 



{1,2} 



Li^l + i^2^{l,2} < ^(^1; ^2^{l,2}l^r^{l,2}) 



^1^{1,2} + Li^i 
^D{Pue^^U'_ 



Pw^ w 



< /(Yi; /7f^|i^2}' ^l^ll^2^{l,2} 
{1,2}) 

< /(Yi; ^i%l|^f^{i,2}' ^2^{l,2}. 



^1^{1,2}^ 

^1^{1,2} + ^2^2 + P2^{1,2} < ^(^2; Ul 

\ ^2^2^2^(1,2} I I ^2^2^2^{1,2} ^ 

^^2^2 + i:2^{l,2} < I{Y2', /72%2, ^|^{l,2}l^r 



^D{Pu 



2^2^{1,2}II ^' 



2^2^2^{1,2} 



{1,2}' ^2^25 ^2^{1,2} 



41,2} 



L2^2 < -'"(^2;C^2^2l'^f^{l,2}'^2^{l,2}) 

+D{PuU\PuUPu^.^,...y) (17) 
union over all the distributions that factor as 



codebook 



= P, 



^1^{1,2} ^1^1 I^1^{1,2} ^1^2 I^1^{1,2} 



encoding 



^1^1^1^2^1^{1,2} ' 



(18) 



-R\- 



for the rate splitting strategy in ([T6l) and Li^j = R\^-^ - 

Proof: The complete proof is provided in (91. ■ 
From the FME of the binning rates i^i^j one obtains that 
the largest achievable region in (^1^1, ^2^2) achieved with 
the choice Pf/e ^ ^ = P^/c and Pt/e ^ ^ = Pjjc . 

^1^{1,2} ^1^{1,2} ^2^{1,2]| ^2^{1,2} 

With this choice the achievable region in Cor. IIII.3I becomes 
equivalent to the Han and Kobayashi region 1 15 1, which is the 
largest known achievable region for a general IPC. 

As for the MAC , Cor. HIUl and the BC, Cor. HIOl Cor. 
nil. 3 1 does not improve on the largest known region for the 
IPC but shows that a larger set of transmission strategies than 
superposition coding and binning can be used to achieve this 
region. 

In |T, Sec. VI] we have shown that superposition coding and 
binning can both be used to achieve the largest known inner 
bound for the MAC, BC and IPC. In the examples above we 
have shown that combining the two encoding strategies into 
a new and more general transmission strategy still achieves 
the same performance. With these considerations in mind, we 
can provide an insight on the error performance of these two 
coding techniques. Both in superposition coding and binning 
one creates multiple codewords to transmit the same message. 
In superposition coding, the message encoded in top codebook 
is associated to multiple codewords, one for each possible base 
codeword, while, in binning, codewords in the same bin are 
associated to the same message. When U2 , with rate R2, 
is superposed io , with rate Ri, the number of possible 
codewords used to encode the same message 'mU2 \^ 2^^^ .In 
binning, the number of excess codewords depends on the joint 
probability distribution between the codewords imposed by 
the encoding procedure. When U2 is binned against Ui, the 
smallest number of codeword in each bin is R2 = I{Ui;U2). 
In both cases, the transmitted codewords [1/1^1/2] belong to 
the typical set 7^ {PU1U2) t>ut superposition coding usually 
requires a larger number of excess codewords than binning 
to achieve the desired typicality property and this number 
is fixed and does not depend on PuiU2- While binning is 
more advantageous than superposition coding at encoding, it 
performs worst at decoding. In superposition coding, after 
the decoding of the base codeword Uf^ , the receiver looks 
for the transmitted top codeword [/2^ in a codebook of 
size 2^^2 jn binning, instead, after has been correctly 
decoded, the possible transmitted codewords are 2^(^2+i?2) 
The knowledge of Uf^ helps the decoder in determining 1/2^ 



in that ^U2 must appear as if generated according to the 
encoding distributions, but it does not reduce the number of 
possible transmitted codewords U2 - Interestingly the encoding 
and decoding benefits provided by superposition coding and 
binning seem to balance each other in the proposed random 
coding technique. 

IV. Conclusion 

In this paper we present a new achievable strategies that 
encompasses superposition coding and binning. The error 
analysis of this new achievable scheme requires a more general 
version of the classical covering lemma that is based on the 
inaccuracy between typical sequences. With this new random 
coding technique we derive achievable regions for the multi- 
access channel, broadcast channel and interference channel. 
These inner bounds do not improve on the largest known 
achievable regions but show that the same error performance 
can be achieved with a large set of encoding strategies. 
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